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ABSTRACT 

The Center for Gifted and Talented Native Hawaiian 
Children has been developing identification methods congruent with 
Hawaiian values and traditional talent areas and has increased the 
number of native Hawaiian students identified and provided with 
enrichment services. This report describes efforts to combine 
self-report data and performance-based assessment to identify 
students. The Center offers a Saturday enrichment program open to all 
native Hawaiians. To identify students for participation in other 
Center programs, especially the summer program, performance-based 
assessment is conducted during the Saturday enrichment program. The 
assessment process involves guided observation using worksheets and 
assessment summaries, training college students in observation ' 
techniques, and encouraging instructors to design enrichment 
activities that allow students to demonstrate their abilities. 
Initial findings indicate a relationship between students 1 
self-reported interests and abilities but unacceptably low 
reliabilities of observers and failure to predict student achievement 
in the summer program. (Contains 15 references.) (DB) 
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Perfomiance-bascd Identification of Culturally Diverse Gifted Students: A Pilot Study 

Educators arc concerned with the undcrrcprcscntation of culturally diverse groups in gifted and 
talented programs (Frasicr, 1987; Gallagher & Kinney, 1874; Patton, Prillaman, & VanTasscl- 
Baska, 1990). In Hawaii, 22% of the public school population is native Hawaiian (individuals 
whose ancestors lived in Hawaii prior to the arrival of Captain Cook in 1778); however, only 11% 
of those participating in state-sponsored gifted programs arc native Hawaiian (State of Hawaii, 
Department of Education, 1988). The Center for Gifted and Talented Native Hawaiian Children has 
been developing identification methods which are congruent with Hawaiian values and traditional 
talent areas and have increased the number of native Hawaiian students being identified and receiving 
enrichment. In this paper, wc describe our most recent efforts in combining self-report data and 
performance-based assessment to identify students for participation in enrichment programs. Wc ask 
the following: To what degree arc students 1 self-reported interests and abilities related to their 
observed performance? What arc the relationships among self-report, observed performance, and 
achievement? Which observed behaviors were the best predictors of success in the enrichment 
program? 

Recently educators have challenged 'traditional' assessment narrowly defined as paper and pencil 
tests and begun examining 'authentic 1 assessment which involves performance of tasks and 
activities valued for their own worth and is consistent with constructivistic assumptions of learning 
(California Assessment Program Staff, 1989; Linn, Baker, & Dunbar, 1991; Shavelson, Baxter, & 
Pine, 1992; Stiggins, 1987; Wiggins, 1989). In gifted education, researchers and practitioners have 
advocated the use of multiple measures in order to identify and assess gifted learners (Clark, 1992; 
Rcnzulli, 1981). While in the >ry, multiple measures are used; in practice, entry to most gifted and 
talented programs is highly dependent upon high scores on achievement and/or intelligence tests 
(Richer!, Alvino, & McDonncl, 1982). This practice may be a factor in the undcrrcprcscntation of 
culturally diverse students in gifted programs (Richcrt, 1987). in order to more validly assess 
students from other than the dominant culture, wc believe that assessment should consist of familiar 
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and culturally relevant situations where procedures and materials arc related to the students' social- 
cultural ecology (Miller- Jones, 1989). Authentic assessment in general and performance-based 
assessment in particular have the potential to contribute to the identification of gifted and talented 
culturally diverse students. 

Since its inception in 1989, the Center has provided Super Enrichment Saturday programs to 
allow bright native Hawaiian students opportunities to discover and enrich their talents. These 
Saturday experiences arc thcmatically developed and provide sessions in multiple talent areas. During 
the first two years, students obtained application packets which included general demographic 
information, specific questions related to highlighted talent areas, and two community/teacher 
recommendation forms. All applications were reviewed by Center staff who removed names and 
prepared the application files for screening committees. Then, specialized screening committees met, 
decided which applicants met the minimum qualifications of high interest and ability, assigned a 
number from 1, definitely admit, to 3, admit if space available, and rank ordered all those who met 
the minimum criteria. The Identification/Assessment coordinator determined how many students at 
each grade level would be accepted, reviewed screening committee recommendations from all 
committees, made final admission decisions, and invited those accepted to participate. Since the 
Saturdays were designed to be inclusive and allow many students opportunities to experience 
enrichment, most applicants who met the minimum criteria were admitted. 

In addition to the Super Enrichment Saturday programs, the Center offers intensive summer 
programs. Initially, information from enrichment sessions was not considered in the identification 
and selection of summer program students. On the basis of evaluation data from two yews of 
programs, changes were made during the 1991/92 school year to open Saturday participation to any 
native Hawaiian student on a first come, first serve basis. In addition to demographic data, 
applications contained interest and ability inventories which provided information used in assigning 
students to specific sessions. The interest and ability indicators were as follows: 

We want to know about your interests. Please rank the following program areas. Use 1 as your firs 
choice; 2 as your second choice; and 3 as your third choice. 
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LEADERSHIP VISUAL ARTS SCIENCE MATH 

(i. c, draw, paint) 

PERFORMING ARTS LANGUAGE ARTS PSYCHOMOTOR 

(i. c, drama, dance, music) (i. c, write, read, speak) (i. c., sporis) 

We want to know about your abilities too. Please circle five of the activities below that YOU CAN 
DO BEST. 

PROBLEM SOLVING SWIMMING TAKING CHARGE KICKING _ 

DRAWING WRITING POEMS PAINTING TELLING STORIES 

EXPERIMENTING COMPUTING CALCULATING ™>^ NG 

WRITING STORIES DANCING READING S NGING 

RUNNING PERSUADING OTHERS THROWING SEEKING RESPONSIBM.I 

MAKING DECISIONS ACTING USING COMPUTERS BOWOTONNY 

SPEAKING LANGUAGES PLAYING NINTENDO ORGANIZING INVENT ING 

In order to identify students for participation in other Center programs, we created a 
performance-based assessment to be used. during the enrichment sessions. 
There arc several key elements of this assessment process. First, observation worksheets and 
assessment summary sheets were developed and piloted in public school classes. The behaviors to 
be observed were adapted from those found in Tuttlc, Becker, and Sousa (1988) and included asks 
questions/demonstrates curiosity, answers or poses solutions to problems, advanced vocabulary, 
concrete level of thinking, abstract level of thinking, knows a lot beyond what you would expect for 
this age, confident, active/ goal directed, interacts most with adult, interacts most with peers. The 
worksheet was designed to record both student and teacher activities (sec Appendix A). Expanded 
descriptions of each characteristic in quick reference form were available on the back left side of the 
worksheet. In addition, there was space for observer comments. The assessment summary form 
contained rating scales for interest and ability in each session and open space lor observers to detail 
Strengths of each student (sec Appendix 13). 

The second key clement was the selection and training of observers. In collaboration with the 
Education Department of the University of 1 lawaii at llilo, students enrolled in educational 
psychology were given an opportunity lo received class credit for being obsei > rrs. The 
Identification/Assessment coordinator and educational psychology instructor designed and conducted 
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training sessions which introduced the forms and provided simulation activities for observation 
practice and debriefing. 

The third critical clement was the content and structure of the Saturday session. In order for the 
assessment to be successful, session instructors had to provide opportunities for students to 
demonstrate their abilities. Therefore, instructors were provided with copies of the observation 
worksheet and assessment summary form and assistance in designing appropriate sessions. 
During the 1991/92 school year, the Center conducted two Super Enrichment Saturdays. One 
hundred sixty-eight native Hawaiian students attended Super Technology Saturday and 237 attended 
Super Creativity Saturday. In order to assess their performance, 30 educational psychology students 
observed from 5 to 10 students through three different sessions which each emphasized a different 
talent area. Students who received summary ratings of 4's or 5's on interest and ability in any one 
talent area (N=13K) were invited to participate in a two week intensive program that integrated visual 
arts, performing arts, language arts, and math/science. Of those invited, 7<S indicated they would 
attend and 70 students actually attended. 

The summer program was interdisciplinary, focused on the 1 lawaiian lhcm< nf lokahi 
(harmony). The four program teachers were selected in language arts, performing arts, visual arts 
and math/science. Activities integrated leadership and Hawaiian culture. In addition, teachers 
emphasized student strengths and encouraged the growth of healthy self-concepts. Throughout the 
two week session, students participated in required and self-chosen activities in a variety of grade 
level and cross age groups. 

The curriculum was designed using the Center model of talent enhancement, self esteem 
building, Hawaiian values, and parent/community involvement, liach day instruction focused upon a 
different subtopic related to lokahi in each of the four disciplines. Subtopics included personal 
lokahi, interdependence, patterns, and the Hawaiian environment. For examine, during the focus on 
patterns students explored patterns in writing, movement, scientific study, and collage making. 
Patterns were presented from the viewpoints of authors, choreographers, biologists, and the graphic 
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designer. On the final day of the program, students participated in a hoike (show/exhibit) that 
demonstrated the concepts, skills, and projects completed during the session. 

Preliminary Results 

As we began examining the data, we felt like Charlie Brown as he and his friends critically 
analyzed the clouds on a perfect spring clay. Alter his friends in great detail reported seeing a variety 
of mythical beasts, Charlie Brown decided not to tell them he saw a bunny and a horsy. We had 
hoped to report in great clarity the relationships between student self-reported interest and ability, 
performance as observed through avaricty of talent areas, and achievement demonstrated in summer 
classes. However, for purposes of this paper only the data from the Super Creativity Saturday (N= 
237) will be discussed. Of the 237 students who attended the creativity sessions, complete 
application and performance data were available for 225 students. Of these students, 43 attended the 
summer program and 24. had complete information on pre/post tests of program affective objectives 
and teacher ratings of interest and ability. In this preliminary analysis, we choose to explore the data 
using correlations (the bunnies and horsics) but as the data set is completed, we will also consider 
multivariate analysis (mythical beasts). 

One of our first questions was, do students' self-reported interests match their reported abilities? 
In other words, if a student (N=225) indicated that he or she was most interested in language arts, 
did that student also select the abilities related to language arts such as writing poem, reading, and 
telling stories? We found the following for each talent area: 



Talent Area 

Leadership 
Visual Arts 



r 

.13 
.40 



Significance 

NS 
.001 



Science/Mathematics .12 NS 

Performing Arts .36 .001 

Language Arts .25 .02 

Psychomotor .51 .001 



Another question was, were the trained observers able to distinguish between interest and ability 
as they summarized student performance data in three different talent areas? The following 
correlation matrix for the pcrformancc-bascd summary data in talent areas assessed were significant 
at the .001 level: 

Interest 

Ability Leadership Visual Arts Science/Math Performing Arts Language Arts 



Leadership 


.96 


-.67 


.65 


-.55 


-.47 


Visual Arts 


-.72 


.92 


-.48 


.75 


.62 


Seicnce/Math 


.64 


-.40 


.96 


-.59 


-.67 


Performing Arts 


-.58 


.69 


-.61 


.97 


.74 


Language. Arts 


-.50 


.58 


-.71 


.73 


.96 
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Wc then asked what is the relationship between self-reported interest and ability and observe*. • 
rated interest and ability? 

Observer Self-Report Interest 



Assessed Leadership Visual Arts Science/Math Performing Arts Language Arts 



Leadership 


.11 




41*** 


-.11 


-.15 


Visual Arts 


-.04 


.28** 


-.34*** 


.08 


.07 


Science/Math 


.03 


-.27** 


33*** 


-.02 


-.14 


Performing Arts 


-.03 


.17 


-.30** 


.07 


.12 


Language Arts 


-.12 


.20* 


-.30** 


.12 


-.16 


Note: .05*. .01**, .001*** 










Observer 




Self-Report Ability 






Assessed 


Leadership 


Visual Arts 


Science/Math 


Performing Arts 


Language Arts 


Leadership 


-.04 


-.26** 


.11 


-.1 1 


.13 


Visual Arts 


.05 




-.13 


.14 


-.01 


Science/Math 


.01 


-.1 


.06 


-.10 


.10 


Performing Arts 


.05 


.19 


-.09 


.15 


.01 


Language Arts 


.03 


.1 


-.14 


.20* 


-.02 



Note: .05*, .01** 

Students who attended the Summer Youth Program were rated by their teachers in interest and 
ability in language arts, performing arts, and seicncc/mathcmaties. We asked if the students' self- 
reported interests and abilities were related to teacher assessment. There were, however, no 
significant correlations. Wc further asked whether there was a relationship between teacher ratings of 
program performance and specific descriptors as reported by the observers. From our sample of 24 
eases, wc found only one significant correlation and it was negative (teacher rated student interest in 
language arts and the descriptor active/goal directed, -.43, p < ,05). 
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Discussion 

Now for the clouds. Any inferences we make at this time arc tentative. It appears thai there is a 
relationship between students' self-reported interests and abilities. That is, students who are 
interested in visual arts, for example, also reported being 'good' at activities in the visual arts. 
However, the self-reported interests and abilities don't always match those identified by observers. 
The exceptions were interest in visual arts (r = .28, p > .01) and mathematics/science (r = .33, p > 
.001) and ability in visual arts (r = .32, p > .01) . From this, it appears that the visual arts might be 
a good starting point for performance assessment in identifying gifted and talented native Hawaiians. 

Observers don't appear to discriminate between interest and ability when summarizing student 
performance so we found high correlations among these ratings. In the future, we will consider a 
single summary estimate that encompasses both interest and ability. Actually, the most helpful aspect 
of the summary sheets were the comments. Although these weren't used to select the students, they 
were used by the summer teachers to plan instructional activities and groups. An avenue for further 
research is how to use the comments in the selection procedure. We arc also concerned with the 
reliability of the observers. In a concurrent study, Javier (paper in progress) is examining the 
interrater reliability of the observers using cases where two observers recorded data for the same 
student. To date, the reliabilities appear to Ik unacceptably low. 

The clouds we have observed do have a silver lining. The Saturday observation cxprrietuv lirs 
provided many prcserviee teachers with an opportunity to view native Hawaiian rhildrcn a-. I'ilVd 
and talented, consider individual differences in talents and abilities, and see u m- ook (lit ones m 
action. The training includes information about gifted and talented students, instruction designed to 
match native 1 lawaiian preferred methods of learning, and issues related to 1 lawaiian education. The 
project has also provided enrichment activities for approximately 350 native 1 lawaiian students and 
opportunities to apply for additional learning experiences through the Center. 

The long term goal of this project is to create a reliable and valid system of performance-based 
identification of native 1 lawaiian students which might then serve as a model for other 
undcrrcprcsentcd groups, From this analysis, it is clear that we have a long way to go. In cxamininj 
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application data, performance assessment data and student achievement in the summer program, we 
have not found high correlations or been able to build a prediction equation. 
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